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(54) A graphical interface 

(57) An electronic test system (1 00) is provided with 
a multi-level tree structure (309) for summarizing the re- 
sults of the test. The levels of the tree correspond to the 
hierarchical levels in the execution of a test procedure. 
The highest level (402), or first level, represents the re- 
sult for the whole test procedure. Level two (404) repre- 
sents the results of various tests. Level three (408) rep- 
resents the results of various measurements performed 



in a test, and level four (41 6) represents the results for 
various datapointsfora given measurement. Preferably, 
each level in the tree structure is characterized by a 
graphical element (402,404,408,416). The graphical el- 
ement provides the information on the status of the re- 
sult at that level. In the tree structure, the result at a high- - 
er level (408) are determined by summarizing the results 
(416,418,420) at the adjacent sub-level. 



CM 
< 
CM 

N- 

CD 
O 
CO 



CL 
LU 



117 



3 



106- 



TEST 
EQUIPMENT 



OUTPUT 
DEVICE 



118- 



116- 



102- 



100 



-114 



MICROPROCESSOR 



DUT 



110 

A 



MEMORY 



-101 



-112 



INPUT 
DEVICE 



-104 



T 

108 



FIG. 1 



Printed by Jouvc, 75001 PARIS (FR) 



DOCID: <EP 1306742A2J_> 



EP 1 306 742 A2 



Description 

[0001] The present invention relates to an electronic 
test system preferably for performing automated tests 
of complex electronic, electromechanical and mechan- 
ical equipment and products. The system includes a tree 
structure for summarizing the results of the test. 
[0002] Test systems are used to check the status and 
performance of complex electronic, electromechanical 
and mechanical products and equipment. Such tests 
can include validation tests which run through the vari- 
ous operations that the device under test (DUT) is ca- 
pable of and records whether each operation was per- 
formed properly; environmental tests which expose the 
DUT to various combinations of temperature, pressure, 
and humidity, and records the results; production tests, 
etc. Generally, both the DUT and the systems providing 
the environmental and other constraints on the DUT are 
controlled electronically. In the last decade or so, com- 
puterized programs which are capable of controlling a 
variety of automated tests, referred to in the art as "test 
executive" programs, have been developed. 
[0003] Tests usually are defined by a set of rules or 
specifications to which the DUT is compared. The rules 
or specifications generally comprise various inputs de- 
fined by electrical and mechanical parameters applied 
to the DUT, such as voltage, current, specified manipu- 
lations of controls and device parts, as well as environ- 
mental parameters under which the test is conducted, 
such as temperature, humidity, pressure, and the time 
period over which a parameter is applied. Each test will 
include many combinations of the parameters applied 
to each element of the DUT, and often will be repeated 
many times. Thus, as equipment and products become 
more complex, electronic test programs have become 
very long and complex, often requiring several days, or 
even a week or more to run a complete test. 
[0004] Test executive programs in the prior art include 
internal test executive programs developed by Agilent 
Technologies and TESTSTAND software developed by 
National Instruments Corporation, which is described as 
a ready-to-run test executive for organizing, controlling, 
and executing automated prototype, validation, or pro- 
duction test systems. The prior art Agilent Technologies 
programs did not use a graphical user interface, and 
thus it was usually tedious to examine the complete re- 
sults of a test. The TESTSTAND software showed the 
results in a spreadsheet structure that was not easy to 
analyze. The prior art software at best was limited to 
showing the results of a given measurement by colored 
test such as "passed" in blue color, "skipped" in yellow 
color and "failed" in red color. Therefore, it takes signif- 
icant time to scan textual results to determine what fur- 
ther action is needed. When a test fails, it is usually nec- 
essary to examine the details of the test further to find 
the source of the failure before taking further action. 
[0005] The present invention seeks to provide an im- 
proved test system. 



[0006] According to an aspect of the present invention 
there is provided an electronic test system as specified 
in claim 9. 

[0007] According to another aspect of the present in- 
5 vention there is provided a method of displaying test re- 
sults as specified in claim 8. 

[0008] The preferred embodiment can provide a test 
executive system in which test results are more easily 
selected and analyzed; by allowing the user to see the 
10 results of the test on a display in a form of a hierarchical 
tree structure. That is, the results are displayed in a mul- 
tilevel structure having a highest level and one or more 
sublevels. In each branch of the tree, the results are 
summarized from each sublevel to the next highest lev- 
's el. That is, if a selected result is recorded at a sublevel, 
this results is passed up to the next level, which displays 
the result. Thus, a high level scan of the test results will 
quickly show the selected results. 
[0009] A tree structure is any structure organized in 
2 o the manner of the branches of a tree. That is, a tree is 
organized as a trunk, to which one or more branches 
are attached. Each branch in turn has one or more sub- 
branches attached to it, and so forth, until we reach the 
level of a twig, which has no additional branches, at- 
25 tached. Whether the "trunk" or the "twig" is considered 
the highest or lowest level is arbitrary. In this disclosure, 
we shall consider the main level, or trunk, the highest 
level, and lower levels will be considered as sub-levels 
of the next higher level. Thus, a tree structure is a hier- 
30 archical configuration having multiple levels, which each 
sub-level having a number of entries equal to or great 
than the next higher level. Preferably, the user can 
"click" on a branch to open a level and see where the 
branch leads. 

35 [0010] The levels of the tree structure preferably cor- 
respond to the hierarchical levels in the execution of a 
test procedure. The highest level, which we shall refer 
to as the first or top level, represents the result for the 
whole test procedure. Level two represents the results 

40 of various tests. Level three represents the results of 
various measurements performed in a test, and level 
four represents the results for various datapoints for a 
given measurement. In summary, the levels of the hier- 
archy, starting from the highest level, are preferably, pro- 

45 cedure, test, measurement, and datapoint. Preferably, 
the non-optimal results are summarized, and most pref- 
erably, the least optimal result is summarized. For ex- 
ample, if the possible results are "fail", "marginal", and 
"pass", with "fail" being the most non-optimal result, and 

50 "marginal" being more optimal than "fail" but less opti- 
mal than "pass", then "fail" is summarized over "margin- 
al" and "marginal" is summarized over "pass". If no non- 
optimal result is found in a branch, then the optimal re- 
sult is summarized up to the highest level. Preferably, 

55 the optimal result is "pass". 

[0011] The electronic test output is displayed through 
an easy-to-use graphical user interface (GUI). The por- 
tion of the display containing the tree structure is pref- 
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erably a small portion of the graphical user interface 
(GUI) where the results of the tests are preferably dis- 
played as graphical elements. Preferably, the results are 
also displayed as textual elements. 
[0012] Each level in the tree structure is, preferably, 
characterized by a graphical element. The graphical el- 
ement provides the information on the status of the re- 
sult at that level. Preferably, the graphical element con- 
sists of a single graphical element, such as an icon. Pref- 
erably, each icon is coded to indicate to the user the sta- 
tus of each result. Preferably, the code is an assigned 
symbol and/or a color. Preferably, each icon is colored 
to help the user instantly evaluate the outcome of each 
test. For example, the color red is assigned to a "fail" 
result. 

[0013] Preferably, the tree structure also includes 
simple alphanumeric information describing the nature 
of each level. The alphanumeric characters are, for ex- 
ample, a text string providing the function or parameters 
of the level. 

[0014] Preferably, as indicated above, the tree struc- 
ture is an expandable tree structure. That is, when the 
tree structure displays only the main branches, the user 
activates a branch by clicking on the icon corresponding 
to that branch, and it expands by showing daughter 
branches which are sub-levels in the execution of the 
test program as previously mentioned. Similarly, by 
clicking again on a branch, the daughter branches are 
hidden and summarized on the next higher level. 
[0015] The preferred embodiment not only provides 
an electronic test program that displays the overall out- 
put in a form that can be easily and quickly evaluated, 
but also displays the output in an intuitive manner which 
permits the user to quickly check individual results. 
[001 6] * Embodiments of the present invention are de- 
scribed below, by way of example only, with reference 
to the accompanying drawings, in which: 

FIG. 1 is a block diagram showing the principal 
hardware components of a preferred embodiment 
of the present invention connected to a device un- 
der test; 

FIG. 2 is a block diagram showing the hierarchical 
structure of the preferred embodiment of test pro- 
gram; 

FIG. 3 shows a view of the graphical user interface 

of the electronic test system; 

FIG. 4 shows an embodiment of tree structure; and 

FIG. 5 shows an alternative embodiment of a tree 

structure. 

[0017] The described embodiment relates to an elec- 
tronic test system with a tree structure for summarizing 
results. Referring to FIG. 1 in the preferred embodiment, 
the test system preferably comprises a computer 100. 
Computer system 1 00 includes a memory 1 01 , a micro- 
processor 102, an input device 104, and an output de- 
vice 106. Memory 101 communicates with microproces- 



sor 102 via electrical line 110. Input device 104 commu- 
nicates with microprocessor 102 through electrical line 
112. Microprocessor 102 outputs the data through out- 
put device 106 via electrical line 114. 

5 [0018] In another embodiment, the test system can 
take the form of software stored in memory 1 01 and run 
or executed by processor 102. The user interacts with 
the test system through input device 104 such as, but 
not limited to, a keyboard, a mouse, a track ball, a touch 

10 pad, or a joystick. The input device 1 04 allows moving 
a cursor or a pointer on the output device 1 06, that is, a 
display system such as a cathode ray tube monitor or a 
liquid crystal display. The results of the test are dis- 
played on the output device 106. The test is controlled 

15 by processor 1 02, which communicates the instructions 
of the test program to the product under test or device 
under test (DUT) 108 via electrical line 116. Processor 
102 controls test equipment 117 via electrical line 118. 
The test results are processed by processor 102 and 

20 stored in memory 101 to be visualized by display 106. 
The display information contains both information as to 
the configuration and position of the cursor and other 
visual information of interest to the user such as the re- 
sults of the test and how the test is progressing. 

25 [0019] The system may be implemented in a variety 
of actual electronic devices that follow the general form 
outlined on FIG 1. For example, the test system may be. 
embodied in a computer system, or it may also be em- 
bodied in the hardware as a logic circuit, or it may be 

30 embodied as an electronic testing device such as, but 
not limited to, an electronic analyzer. In some embodi- 
ments, no test equipment is required, so test equipment 
17 is not present. 

[0020] To better understand the workings of the sys- 

35 tern, it is helpful to describe the hierarchical structure of 
the preferred test program and the order in which the 
test is performed. Referring to FIG. 2, there is shown a 
block diagram 200 illustrating the hierarchical, i.e., mul- 
ti-level, characteristic of the test program. The first level 

40 201 corresponds to the product model , a file the test de- 
veloper creates to test a family of specific device model 
numbers. It contains test procedures and inputs. 
[0021] The next level 202 corresponds to the proce- 
dure itself. A procedure is an ordered list, sequence, or 

45 script of tests to be run. Several procedures may exist, 
which is represented in FIG. 2 by a deck of cards 202, 
each representing a different procedure. Each proce- 
dure includes a plurality of tests, i.e., test 1 , test 2, test 
3... test N, as shown at 203. Each test includes a plurality 

50 of measurements. This is illustrated in FIG. 2 for test 2 
shown at 205. As shown, test 205 includes measure- 
ments 207, i.e... measurement 1. measurement 2... 
measurement N. Each measurement includes one or 
more datapoints : each datapoint represented as one of 

55 the cards 210, 211, 212 in a stack of cards 214 associ- 
ated with each measurement. A procedure is defined by 
writing a program or a code to build a structure of soft- 
ware objects. In one embodiment, the software objects 
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are Component Object Model (COM) objects. COM is a 
language-independent component architecture, not a 
programming language. It is meant to be a general pur- 
pose, object-oriented means to encapsulate commonly 
used functions and services. See Newton's Telecom 
Dictionary by Harry Newton, Publishers Group West, 
page 1 97. 

[0022] A test 205 is a group of measurements 207 in 
a procedure 202 that share the same test algorithm or 
the same test software code. Some examples of tests 
include amplitude accuracy test, test of harmonic distor- 
tion, etc. The test program repeatedly calls a test for 
each measurement and datapoint. 
[0023] A measurement, such as measurement 206, 
is a configuration or a set up for a test. Each measure- 
ment, in measurements 207, within a test 205 can have 
different setups or configuration parameters. Tests are 
parameter driven and the parameters are inputs at the 
measurement level. Measurements are elements such 
as range in volts, frequency in kilohertz or harmonic (an 
integer number). The test procedure 202 views the 
measurements 207 as data to be passed from the pro- 
cedure to a test. A measurement is also a phase of test 
execution. During the measurement phase of test exe- 
cution, the measurement is started but data is not col- 
lected. This allows for multiple devices under test 
(DUTs) to be configured and triggered together. 
[0024] A datapoint, such as 210, 21 1 , 212, is a subset 
of a measurement, such as 206, containing additional 
parameters that select a result when one measurement 
generates multiple results. Some examples of multiple 
datapoints for a measurement are the minimum and 
maximum of a spectrum analyzer sweep or each chan- 
nel of a device. 

[0025] For each datapoint, such as 210, in measure- 
ment 206, a value result is extracted. The results ob- 
tained are compared to specifications. Specifications 
are numerical limits, string match, or Boolean pass/fail. 
There are three sets of limits: marginal limits, production 
line limits, and customer limits. Each limit has an upper 
and a lower value. 

[0026] All the inputs and outputs of the electronic test 
system (FIG. 1) are handled preferably through a graph- 
ical user interface. FIG. 3 shows the graphical user in- 
terface 300 as displayed on the output device 106. The 
"tape recorder type buttons 301 on the top left are used 
to control the test. From left to right, the buttons are: 
abort 302, restart test 303, restart measurement 304, 
pause 305, run 306 : skip measurement 307, skip test 
308. 

[0027] The right side 314 of the graphical user inter- 
face 300 shows a window 340 comprising a series of 
rows, such as 315, and columns, such as 316. Window 
340 displays the time 31 7 the test is run, as well as the 
status of the test 31 8. This window also displays the test 
that is being carried out, such as amplitude accuracy in 
319, the type of measurement, such as (range=5Vp, 
Frequency=1 kHz) in 320, the datapoint or channel un- 



der test, such as (Ch=1 , Ch=2, Ch=3) in 321 , the value 
or result of the measurement, such as (0.1235 dB) in 
322, the specification, such as (± 0.2) in 323, and the 
frequency, such as (1 kHz) in 324. The test results can 

5 be sorted by clicking on the "view" buttons in window 
325 above the window 340. This allows either all the test 
results to be displayed, or permits the results to be fil- 
tered and displayed depending on their marginal or 
failed status. The bottom right window 330 is a progress 

to window indicating the progress of the test procedure. 
[0028] On the left side of the graphical user interface 
(GUI) 300, the preferred embodiment of a tree window 
309 is shown. Window 309 includes a hierarchy of tests, 
measurements, and datapoints. Icons corresponding to 

*5 the results indicate pass, fail, marginal and not-yet-test- 
ed. "Smiling" 310, "surprised" 311 , and "sad" 312 icons 
correspond respectively to the pass, marginal, and fail 
status. The pass/fail icon 31 3 on top of the window is for 
the entire procedure. It summarizes the status of all the 

20 tests with "fail" having priority. That is, if there is one 
failed test, the whole procedure is fail. As discussed in 
more detail below, an algorithm that promotes the least 
optimal results is used to compute the final status of the 
procedure. 

25 [0029] An enlarged view of tree window 309 is shown 
in FIG. 4. The tree structure contains multiple levels with 
the first or highest level corresponding to the final test 
procedure shown with icon 402. This is considered the 
trunk of the tree structure. The second level in the exe- 

30 cution of the test procedure comprises one or more 
tests, such as 404, each including one or more meas- 
urements, such as 408. In the embodiment shown, the 
second level includes two tests: an amplitude accuracy 
test shown with its corresponding result icon 404, and 

35 a harmonic distortion test shown with its result icon 406; 
this test level corresponds to the first generation branch- 
es extending from the trunk. The third level in the exe- 
cution of a test represents the type of measurement it- 
self, such as (range=5Vp, Freq=1 Khz) with its corre- 

40 sponding result icon 408 : (range=5Vp, Freq=100kHz) 
with its corresponding result icon 41 0, (range=5 Vp, Har- 
monic=2) with its corresponding result icon 412, and 
(range=5Vp, Harmonic=3) with its corresponding result 
icon 414. This level corresponds to the second-genera- 

45 tion branches in the tree. The fourth level in the execu- 
tion of the test procedure represents datapoints, such 
as (Ch=1 , Ch=2, Ch=3) with their respective result icons 
416, 418, and 420, associated with the datapoints 
(Ch=1, Ch=2, Ch=3) being sub-levels of measurement 

50 408, and respective result icons 422, 424, 426 associ- 
ated with the (Ch=1 , Ch=2, Ch=3) datapoints being sub- 
levels of measurement 41 0. This level corresponds to 
the third-generation branches in the tree. 
[0030] Preferably, the tree structure 309 is expanded 

55 or retracted by "clicking" on a "V or "-" sign next to the 
icon corresponding to the result for each level, such as 
the sign 430 next to result icon 404, the "-" sign 435 
next to result icon 408, the "-" sign 440 next to result 
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icon 410, the "-" sign 445 next to result icon 406, etc. 
Initially, the tree structure is preferably retracted to only 
display the trunk; that is, the result for the final test pro- 
cedure 402 and the results for the first-generation 
branches, such as 404 and 406, respectively, amplitude 
accuracy test and harmonic distortion test. If the sign 
next to the result icon is ,, +", such as 450 and 455, this 
means that daughter branches, i.e., sub-levels, are not 
displayed. However, the user can activate the tree struc- 
ture in order to show more detail as to where each 
branch leads by "clicking" on the V sign next to the 
result for the parent branch. If the sign next to the result 
icon is such as 430, 435, 440, 445, this means that 
daughter branches are displayed and the user can ex- 
pand further the tree to explore next generation branch- 
es. 

[0031] As previously mentioned, the result for each 
level or branch in the tree structure is indicated by an 
easily readable icon. Preferably, a "smiling face", such 
as 416, is selected to indicate a "pass" result, a "sad 
face", such as 404, is selected to indicate a "fail" result, 
a "surprised face", such as 410, is selected to indicate 
a "marginal" result, and a "?" is selected to indicate a 
"not-yet-tested" result. Most preferably, each result icon 
is color coded to indicate the status of the result. For 
example, the color green is assigned to a "pass" result, 
the color red is assigned to a "fail" result, the color yellow 
is assigned to a "marginal" result, and no color is as- 
signed to a "not-yet-tested" result. 
[0032] Preferably, the result for a level is determined 
by applying an algorithm where the least optimal result, 
e.g., "fail" is given precedence over optimal results, e. 
g. t "pass". 

[0033] Another way of expressing the summarizing 
function is that, in the tree structure, the results are sum- 
marized to the higher levels. That is, if a test fails at a 
datapoint level, for example, that failure is shown at the 
measurement level, the test level, and the procedure 
level. The user can investigate where a fail result origi- 
nated by following the results in each branch directly 
connected to the shown result. In practical terms, for ex- 
ample, the user sees a fail result in final procedure 402 
and would like to investigate where the fail result origi- 
nated. The user opens the first generation branches 404 
and 406. Icon 404 shows a fail result and icon 406 a not- 
yet-tested result. Therefore, the user decides to lookfur- 
ther along the branch where the fail in icon 404 originat- 
ed. The user opens 404 by clicking on the 'V sign next 
to it to show daughter branches 408 and 41 0. Icon 41 0 
shows a marginal result and icon 408 shows a fail result. 
Therefore, the user opens branch 408 to show results 
416, 418, and 420. The user sees a fail result at 420. 
The user, therefore, determined that the fail result orig- 
inated from a fail result in 420, that is, a fail result in 
datapoint (Ch=3) under measurement (range=5Vp, 
Freq=1 kHz) under amplitude test accuracy. 
[0034] FIG. 5 shows an alternative embodiment of 
tree window 500. 



[0035] Referring to FIG. 5, 500 represents another 
form for a tree structure wherein the trunk of the tree, 
also called the first level, is displayed as the cosmic uni- 
verse shown with its corresponding result icon 502. The 

5 second level in the execution of the test procedure is a 
test. This level is displayed as a galaxy, such as 504, 
506, and 508. The icon 510 at the center of galaxy 504, 
the icon 512 at the center of galaxy 506, and the icon 
514 at the center of galaxy 508 represent the results at 

10 the test level. The third level in the execution of a test 
procedure represents the measurement itself with its 
corresponding result icons, such as 516 and 518 in test 
504. This level is represented by stars displayed as cir- 
cular icons such as 516 and 51 8 in the arms of galaxy 

15 504. The fourth level in the execution of the test proce- 
dure represents datapoints in a measurement, with their 
result icons such as 520, 522, and 524 in measurement 
51 8. This level is displayed as planets such as 520, 522, 
and 524 gravitating around star 518. The size of the 

20 icons representing the results at each level increases 
with the hierarchy in the test. For instance, the icon 510 
corresponding to the result at the test level is bigger that 
the icon 51 8 corresponding to the result at the measure- 
ment level. Furthermore, as described previously, icons 

25 are coded to indicate pass, marginal, fail, or not-yet-test- 
ed results at each level in the execution of the test pro- 
cedure. The result for the whole test procedure is also 
color coded and displayed on the icon 502. The user 
can instantly detect and locate where a fail result origi- 

30 nated by looking at this picture "universe". 

[0036] There has been described what at present is 
considered to be the preferred embodiment of the in- 
vention but other embodiments are possible. For exam- 
ple, while the system has been described in terms of an 

35 electronic test program with a tree structure, other sys- 
tems may be implemented based on hardware or 
firmware instead of software. The program can, for in- 
stances, sit on a programmed processor or other plat- 
form. In addition, more than one tree structure can be 

40 added to the graphical user interface to display the test 
results of two or more devices under test. The tree struc- 
ture may be displayed in other forms, such as where the 
results at each level of the tree structure may be 
grouped in another form such as a cosmic universe con- 

45 taining galaxies which in turn contain stars which con- 
tain planets. The present embodiment is, therefore, to 
be considered as illustrative and not restrictive. 
[0037] The disclosures in United States patent appli- 
cation No. 10/000,600, from which this application 

50 claims priority, and in the abstract accompanying this 
application are incorporated herein by reference. 



Claims 

55 

1 . An electronic test system (1 00) including: a memory 
(1 01 ) for storing a test and test results; an electronic 
processor (102) communicating with said memory 
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for controlling the execution of a test and displaying 
the test results; and a graphical user interface out- 
put device ( 1 06) communicating with said electronic 
processor for displaying information, said informa- 
tion comprising test results organized into a tree 
structure (309) comprising a hierarchical configura- 
tion having multiple levels including a highest level 
(402) and one or more sub-levels (404.408,416), 
with the results of each sub-level summarized to the 
next higher level. 

2. An electronic test system as in claim 1, wherein 
each level in said tree structure includes a graphical 
element (402,404,408 : 41 6) representing the status 
of the result at each said level. 

3. An electronictest system as in claim 2, wherein said 
status is one of the following states: pass (41 6), fail 
(408), marginal (426), and not-yet-tested (412). 

4. An electronic test system as in claim 2 or 3, wherein 
said graphical element includes a code selected 
from the group consisting of symbols, alphanumeric 
characters, design and colors. 

5. An electronic test system as in any preceding claim, 
wherein said result at each of said levels is deter- 
mined by applying a summary of each said sub-lev- 
el of said level. 

6. An electronic test system as in any preceding claim, 
whereon said levels include a test procedure level 
(402), a test level (404) which is a sub-level of said 
test procedure level, and a measurement sub-level 
(408) equivalent to a second generation branch in 
said tree structure, and wherein said test procedure 
level is said highest level, equivalent to a trunk in 
said tree structure, said test sub-level is equivalent 
to a first generation branch in said tree structure, 
and said measurement sub-level is equivalent to a 
second generation branch in said tree structure. 

7. An electronic test system as in any preceding claim, 
wherein said tree structure is expandable to display 
said one or more sub-levels. 

8. A method of displaying the results of an electronic 
test including the steps of: performing a test proce- 
dure to determine a plurality of test results and dis- 
playing said test results in a tree structure (309) in- 
cluding a hierarchical configuration having multiple 
levels includes a highest level (402) and one or 
more sub-levels, (404,408,416); and summarizing 
said results in each said sub-level to the next higher 
level in said tree structure. 

9. A method as in claim 8, including the step of dis- 
playing a test string in each level, said test string 




>6 742 A2 1 0 

defining the function of each level. 

10. A method as in claim 8 or 9 : wherein said step of 
summarizing corresponds to summarizing the least 
5 optimum results at said sub-levels to said higher 

level. 
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